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ABSTRACT 


Background: Heart disease is the number one killer of both men and women in the United States, yet a comprehen¬ 
sive and evidence-based heart disease knowledge assessment is currently not available. Purpose: This paper describes 
the two-phase development of a novel heart disease knowledge questionnaire. Methods: After review and critique of 
the existing literature, a questionnaire addressing five central domains of heart disease knowledge was constructed. 
In Phase I, 606 undergraduates completed an 82-item questionnaire. In Phase II, 248 undergraduates completed a 
revised 74-item questionnaire. In both phases, item clarity and difficulty were evaluated, along with the overall fac¬ 
tor structure of the scale. Results: Exploratory and confirmatory factor analyses were used to reduce the scale to 30 
items with fit statistics, CFI = .82, TLI = .88, and RMSEA = .03. Scores were correlated moderately positively with 
an existing scale and weakly positively with a measure of health literacy, thereby establishing both convergent and 
divergent validity. Discussion: The finalized 30-item questionnaire is a concise, yet discriminating instrument that 
reliably measures participants’ heart disease knowledge levels. Translation to Health Education Practice: Health 
professionals can use this scale to assess their patients’ heart disease knowledge so that they can create a tailored 
program to help their patients reduce their heart disease risk. 
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BACKGROUND 

For over 80 years, heart disease has been 
the leading cause of mortality for both men 
and women in the United States. 1,2 In 2005, 
heart disease claimed the lives of more than 
860 000 Americans. 3 Coronary heart disease 
(CHD), the most common type of heart 
disease in the U.S., 4 is responsible for about 
500 000 deaths per year. 3 Results from the 
National Heart, Lung, and Blood Institute’s 
Framingham Heart Study found that the 
lifetime risk for developing CHD at age 40 is 
a l-in-2 chance (48.6%) for men and l-in-3 
chance (31.7%) for women. 4 Moreover, it is 


estimated that every 30 seconds an American 
will suffer from a coronary event and every 
60 seconds the event is fatal. 3 

Not only is heart disease the number 
one killer of Americans, but some of the 
disease’s modifiable risk factors comprise the 
leading three direct contributors to death in 
the U.S.: tobacco use, poor diet and physical 
inactivity, and alcohol consumption. 5 Other 
modifiable risk factors for developing heart 
disease include obesity, high blood pres¬ 
sure, high blood cholesterol, and stress. 6 
Thus, much of an individual’s risk of heart 
disease—and likewise, of death—can be 
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mitigated by addressing one or more of these 
modifiable factors through individual means 
such as by quitting smoking, exercising, im¬ 
proving diet, or obtaining medical treatment 
for conditions such as diabetes, high blood 
pressure, or unhealthy blood cholesterol. 

Heart disease will continue to be a 
modern-day health threat unless the Ameri¬ 
can public has a clear and comprehensive 
understanding of what, and how, risk fac¬ 
tors contribute to the development of the 
disease. Further, individuals need not only 
be able to identify their risk factors, but also 
to understand them in the context of overall 
heart health knowledge. Ensuring that indi¬ 
viduals possess a basic understanding of the 
disease itself—including its symptomatol¬ 
ogy, as well as related medical and dietary 
knowledge—will enable them to identify 
specific actions they can take to reduce their 
risks and to build a foundation upon which 
they can effectively take control of their 
health. Hence, having the ability to assess 
peoples’ level of knowledge about heart 
disease through a standardized tool would 
be valuable in aiding the design of effective 
health interventions, as well as in measur¬ 
ing the efficacy of such interventions. In 
addition, such a tool could effectively serve 
as a means by which to tailor educational 
materials to individuals directly, to address 
their specific gaps in knowledge. 

PURPOSE 

The goal of the current study was to 
develop an easily-administered and psycho- 
metrically sound heart disease knowledge 
questionnaire that could be used in a 
variety of adult populations. To achieve 
our purpose, we identified and extracted 
selected items from existing scales, merging 
them with newly developed items based 
on current cardiovascular research. The 
current scale encompasses five relevant 
knowledge domains: dietary knowledge, 
epidemiology, medical information, risk 
factors, and heart attack symptoms. Sources 
for the existing and new items are reported 
in the Methods section. 

The questionnaire was developed in two 
phases. The first phase (Phase I) was explor¬ 


atory; we developed and tested a paper and 
pencil 84-item questionnaire with a univer¬ 
sity student sample. The primary goals were 
to validate the questionnaire’s five-domain 
structure and to reduce the number of items. 
We also included additional items that evalu¬ 
ated participants’ perceptions of their heart 
disease knowledge. In Phase II, we validated 
a refined version of the scale with a second 
university student sample. 

We identified existing measures of heart 
disease knowledge through a search of 
literature published as of February 2010 
using PubMed and PsycINFO databases 
and Google Scholar. Articles reviewed were 
restricted to those written in English, with 
no limits on year of publication. We used 
keyword search combinations such as: 
“heart health” and/or “heart disease” with 
“knowledge,” “development of,” “test,” 
“scale,” ’’questionnaire,” and “assessment” to 
find related articles. We did not review scales 
that included a majority of open-ended or 
Likert-scale type questions where there was 
not an obvious correct answer 7 ' 11 or scales 
that focused specifically on detection, as 
opposed to more general knowledge of 
heart disease. 12 We also excluded from the 
review any studies that included participants 
aged 18 and younger, 10,13,14 since the current 
study’s samples comprised of participants 
18 and older. Sixteen scales ultimately met 
our criteria for review. 

Each of the 16 scales was examined using 
a set of criteria developed by the authors that 
were based on scale construction, adminis¬ 
tration and scoring. Within scale construc¬ 
tion, degree of comprehensiveness (evalu¬ 
ation of how many knowledge domains 
were addressed), degree of generalizability, 
understandability and suitability of wording, 
degree of difficulty of the questions, and 
total number of items were evaluated. Scale 
administration review involved assessing 
mode of administration, as well as sample 
group and sample size (i.e., whether the 
scale could be used on multiple population 
samples and was tested on a large enough 
sample size). Finally, our assessment of scale 
scoring took into account incorporation of 
open-ended items and use of continuous 


response scales (e.g., Likert-type scales). 
Our review process revealed many issues 
(addressed in the critique below) with the 
existing scales and led us to believe that our 
current scale would not only address these 
issues, but add to the field of cardiovascular 
disease patient education by providing a 
more comprehensive and updated scale. See 
Table 1 for summary of findings. 

Many scales only addressed a subset of 
the five domains. 15 ' 19 Others were tailored 
specifically either to participants in an 
intervention program or to patient sub¬ 
populations. 16,17,20 ' 23 In other cases, item 
wording was outdated or the use of termi¬ 
nology was too technical. For example, scales 
that contain terms such as “angina pectoris,” 
“atherosclerosis” 24,25 and “myocardial infarc¬ 
tion” 18,23 may not be practical for those with 
low levels of education. 

Some scales may also lead to ceiling ef¬ 
fects in performance. For example, among 
the 25 items used in the Heart Disease Fact 
Questionnaire, 22 18 items were found to 
have a mean difficulty score of .80 or above, 
meaning that the majority of the items were 
answered correctly 80% of the time by par¬ 
ticipants. Without inclusion of items having 
a broader range of difficulty, such scales do 
not allow for the detection of meaningful 
differences in knowledge. Further, many 
scales contain too few or too many items. A 
scale with too few items may not adequately 
measure individuals’ knowledge levels with 
sufficient sensitivity, and may also raise 
content validity concerns. 24,26,27 Conversely, a 
scale with too many items increases respon¬ 
dent burden and may render it impractical 
for use in settings in which time may be 
limited, such as during visits with health 
practitioners. 25,28,29 

The modes by which some scales were 
administered may have generated invalid 
results, such as those conducted via tele¬ 
phone 27,29 or otherwise in the form of an oral 
interview. 18,27 Administering a scale verbally 
introduces the possibility of interviewer bias 
or inconsistency. Further, respondents are 
also more likely to answer in socially desir¬ 
able ways when surveyed orally. 31 Another 
problem is utilization of a small sample. 29 
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Table 1. Criteria Met by the Previous Scales 


Authors 

Year 

Scale Construction 

Comprehensive 

Generalizability 

Wording 

Item Difficulty 

Collins et al. 

2004 


x a 

X 

NR b 

DeForge et al. 

1998 



X 

NR 

Farquhar et al. 

1990 

X 

X 


NR 

Folsom et al. 

1988 

X 

X 

X 

X 

John etal. 

2009 



X 


Momtahan et al. 

2004 


X 


X 

Mosca etal. 

2000 

X 

X 

X 

NR 

Mosca et al. 

2004 

X 

X 

X 

NR 

Oliver-McNeil &Artinian 

2002 

X 

X 

X 

NR 

Smith et al. 

1991 

X 



NR 

Suminski et al. 

1999 

X 

X 

X 


Tate & Cade 

1990 


X 

X 


Thanavaro et al. 

2006 

X 

X 


NR 

Thanavaro et al. 

2010 

X 


X 

X 

Wagner et al. 

2005a 

X 


X 

NR 

Wagner et al. 

2005b 

X 


X 



Notes: 

a An “x” indicates criterion met by a particular scale. 

b Not reported “NR” indicates that item difficulty criterion was not reported or that it was not reported for all items. 
c An “x” indicates articles that did not include open-ended or continuous response items. 
d An “x” indicates articles that did include the answer option “I don’t know.” 


Scales which utilize open-ended or con¬ 
tinuous response items are challenging for 
the administrator to score. 18,27 ' 30 Open-ended 
questions elicit a wide range of responses 
and as such, are difficult to score objec¬ 
tively. 32 The use of items with continuous 
response scales renders determining “cor¬ 
rect” answers problematic, because no clear 
correct response exists. Further, several 
scales were also found to contain true/false 
items, but did not include the “I don’t know” 
option as a choice. 16,17,19,20 ' 23,24 ' 26,27,30 Including 
this option helps to reduce guessing. 32 

Our aim was to address these issues via 
construction of a novel scale—one based 
upon the most current medical knowledge 
about heart disease and its risk factors, epi¬ 


demiology, and symptoms that would add 
to the field of cardiovascular disease patient 
education. We did so using two samples 
and cross-validated the resulting scale by 
correlating it with a previously used scale, 26 
as well as a common measure of health 
literacy, 33 to establish both convergent and 
divergent validity. 

METHODS 

Instrument Development 

The current scale encompasses five rel¬ 
evant knowledge domains: dietary knowl¬ 
edge, epidemiology, medical information, 
risk factors, and heart attack symptoms. 
These current domains were derived from 
our literature review of previous scales, the 


self-regulatory model of illness behavior lit¬ 
erature, 34,35 and the findings from an ad hoc 
expert panel consisting of a board certified 
internist, a health psychologist, and psy¬ 
chometrician. Based on our findings, these 
five domains are of equal importance in the 
realm of heart disease knowledge. 

The self-regulatory model of illness 
behavior stipulates that individuals make 
health decisions over three stages: cognitive 
representations stage, action plan stage, and 
appraisal stage. 36 The cognitive representa¬ 
tion stage is defined by five dimensions 
of the health risk: identity/characteristic 
symptoms, cause, consequences, duration, 
and control. 36 ' 39 We believe that our do¬ 
mains, and their associated items, conform 
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Table 1. Criteria Met by the Previous Scales 



Administration 

Scoring 

# of Items 

Mode 

Sample 

Open-ended/ Continuous Response 6 

"1 don't know" d 

X 

X 

X 

X 

X 

X 

X 

X 

X 



X 

X 

X 




X 



X 

X 

X 

X 


X 


X 


X 



X 


X 

X 


X 




X 



X 


X 

X 

X 



X 

X 

X 


X 

X 

X 

X 


X 

X 

X 

X 

X 

X 

X 

X 

X 


X 

X 

X 

X 

X 

X 

X 

X 

X 

X 


to this stage of the self-regulation model 
of illness. 

The selection of existing items and de¬ 
velopment of new ones were based on how 
the items fell within the five pre-established 
knowledge domains. The process was un¬ 
dertaken by the authors, including a health 
psychologist, a research psychologist and a 
psychometrician. Some items were extracted 
from existing, yet dated scales, such as the 
Cardiovascular Disease (CVD) Knowledge 
Test, 26 while others were developed using 
findings and information gleaned from the 
Pittsburgh Healthy Heart Study, American 
Heart Association, Harvard Center for Can¬ 
cer Prevention, Women’s Heart Foundation, 
Heart Healthy Women, National Heart, Lung 


and Blood Institute, National Center for 
Health Statistics, Department of Health and 
Human Services, United States Department of 
Agriculture, Centers for Disease Control and 
Prevention, Mayo Clinic, University of Mary¬ 
land Medical Center, Harvard Medical School, 
and Coalition of Labor Union Women. 

The true-false format was chosen over 
the multiple-choice item format for the 
current scale because we wanted to reduce 
participant burden, given that there were 
over 80 items in the original questionnaire 
(see Phase I instrument description). The 
“I don’t know” option was included to help 
improve the scale’s reliability to reduce 
guessing that is often associated with the 
true-false format. 32 


An ad hoc panel was created to address 
and review the content validity and face 
validity of the domains and items selected 
for the finalized questionnaire (as seen in 
Phase II methods and results). The group 
was composed of three experts in cardio¬ 
vascular disease, health psychology and 
psychometrics. The three experts were asked 
to independently check to see if the items 
had the correct response, fit under the pre- 
established domain, and were up-to-date. 

Phase I Questionnaire 

Instrument description. The initial ques¬ 
tionnaire consisted of 82 items that measured 
dietary knowledge (N = 20), epidemiology 
knowledge (N = 8), medical knowledge (N 
= 24), risk factors knowledge (N = 20), and 
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2 - 

heart attack symptoms knowledge (AT =10). 
Additionally, participants completed three 
items that measured their self-perceived 
degree of knowledge about heart disease: “I 
know more than the average person about 
heart disease,”“I believe that I answered 75% 
or more of the above questions correctly,” 
and “I believe that I answered more of the 
above questions correctly than the average 
person will.” Participants also reported age 
and gender. 

Participants. Participants were 606 
University of Pittsburgh undergraduates 
enrolled in an introductory psychology 
course. Participants were 56.6% female with 
a mean age of 18.67 (SD = 1.66, range = 17 
to 44). Three participants did not provide 
demographic information. Participation 
fulfilled a course requirement. 

Procedure. Participants completed the 
scale as part of a mass testing session. 
The instructions for the heart disease 
questionnaire asked the participants to 
answer “true,” “false,” or “I don’t know” to 
a number of questions addressing their be¬ 
liefs and knowledge about various aspects 
of heart disease. 

Data preparation. From the original 82 
items, we excluded 9 items—8 of which 
were no longer accurate based on new find¬ 
ings in the medical literature at the time 
of analysis (for example, “Lack of physical 
activity is the principal cause of obesity’’ is 
incorrect since it is now understood that 
no 1 principle cause of obesity exists) 40 and 
1 item due to ambiguity (“Men and women 
experience the same symptoms of a heart 
attack .” Women often experience different 
sets of symptoms, but many are symptoms 
common to both men and women). 41 We 
also excluded from analysis two items that 
had a mean difficulty score (% participants 
answering correctly) of .06 or below or .94 or 
above. The remaining 71 items were divided 
among each knowledge domain as follows: 
dietary knowledge (N = 16), epidemiology 
knowledge (N= 8), medical knowledge (N= 
22), risk factors (N = 17), and symptoms (N 
= 8). Within any one domain, less than 2% 
of data was missing. Participants’ answers 
were coded either 0 = incorrect answer or 


“I don’t know” and 1 = correct answer for 
each item on the scale. 

The data from the 594 participants who 
answered at least 68 of the 71 items (95%) 
as well as all 3 items about self-perceived 
knowledge were analyzed. The overall scale 
score was calculated as the sum of the correct 
answers, with percentage score derived by 
dividing each score by 71. We used list-wise 
deletion for missing values. 

Analysis plan. The main purposes of 
Phase I analyses were to evaluate the factor 
structure of the questionnaire and item 
reduction. Our goal was to produce a ques¬ 
tionnaire that contained a reduced number 
of items, but retained the breadth of do¬ 
mains regarding heart disease knowledge. 

SPSS Version 17 was used to obtain de¬ 
scriptive statistics on the overall score of the 
questionnaire and the three items that mea¬ 
sured self-perceived degree of knowledge. 
Upon item reduction (explained below), 
the correlation between the participants’ 
overall scores and scores on each of the three 
perception items was calculated. 

The statistical software data analysis pro¬ 
gram MPlus Version 5.21 ran the confirma¬ 
tory and exploratory factor analyses, given 
its ability to model items with dichotomous 
response format. Two confirmatory factor 
analyses (CFA) were conducted; a five-factor 
solution reflecting the theoretical structure 
of the questionnaire and a one-factor solu¬ 
tion representing all items as measuring a 
general knowledge of heart disease. To assess 
model fit Comparative Fit Index (CFI), 42 
Tucker-Lewis Index (TLI), 43 and Root Mean 
Square Error of Approximation (RMSEA) 
were reported. 44 We attempted to identify 
sources for misfit if the CFA model did not 
fit one or more of the model fit criteria (i.e„ 
CFI > .95, TLI >. 95, RMSEA <. 05). For 
example, we examined the model modifi¬ 
cation indices to identify items that loaded 
on more than one domain. An exploratory 
factor analysis (EFA) was also conducted to 
look for the number of factors and patterns 
represented in the observed data by examin¬ 
ing both the magnitude of the eigenvalues 
and interpretability of the data associated 
with each extracted factor. 


After comparison of the alternative 
one- and five-factor models, item reduction 
began by considering the difficulty of the 
items (assessed by looking at mean scores), 
and the discrimination of the items (assessed 
by looking at factor loadings and item-total 
score correlations), inter-item correlations 
(not reported in text; please contact corre¬ 
sponding author for them), and contribu¬ 
tion of each item’s content towards overall 
knowledge were taken into consideration 
when selecting items for the questionnaire. 
For difficulty, we wanted to ensure that we 
had items that tapped into different levels 
of heart disease knowledge; this guarantees 
that we could discriminate amongst indi¬ 
viduals who may have limited knowledge 
from those who have advanced knowledge. 
For discrimination, we wanted items that 
were related to the overall construct of heart 
disease knowledge to maximize our ability to 
differentiate among individuals at different 
skill/knowledge levels. 

In some cases, an item with a low factor 
loading was retained upon having been 
deemed to measure a key concept not re¬ 
flected in other questions. We also took into 
consideration mean scores reflected in our 
data to ensure the finalized questionnaire 
had a broad enough range of difficulty to 
assess different knowledge levels. 

Phase II Questionnaire 

Instrument description. This instru¬ 
ment was a revised version of the Phase I 
questionnaire. The updated scale excluded 
8 items from Phase I that were no longer 
accurate based on new findings in the medi¬ 
cal literature. The item that was discarded 
for analysis in Phase I due to ambiguity 
was rewritten and included in the Phase II 
questionnaire, as were the 2 items which 
had been excluded from analysis due to 
having had poor item-difficulty scores. The 
resulting scale consisted of 74 items. Par¬ 
ticipants also completed 11 items from the 
CVD Knowledge Test 26 (3 items from this 
14-item scale were already included in the 
scale), 3 items that measured self-perceived 
degree of knowledge about heart disease (the 
same items from Phase I), and 7 items that 
measured health behaviors to see if there 
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was a relationship between health behavior 
status and heart disease knowledge level. A 
final item measured participants’ perceived 
health literacy using a conventional 1 item 
scale, “How confident are you filling out 
medical forms by yourself?” 33 Participants 
also reported age and gender. 

Participants. Participants were 248 
University of Pittsburgh undergraduates 
enrolled in an introductory psychology 
course. Participants were 50.4 % male with 
a mean age of 18.68 (SD = 1.08, range = 
17 to 27). Participation fulfilled a course 
requirement. 

Procedure. Participants completed the 
scale as part of a larger questionnaire in 
an unrelated research study. Participants 
scheduled individual appointments with 
an undergraduate research assistant and 
completed the questionnaire privately on 
a computer in the laboratory. The session 
lasted about one hour. The instructions for 
the Phase II questionnaire were identical to 
those of Phase I. 

Data preparation. From the original 
74 items, three items had mean difficulty 
scores higher than .94 and were discarded 
from analyses. Participants’ item answers 
were coded either 0 = incorrect answer or 
“I don’t know” and 1 = correct answer. The 
scores of 242 participants were used for 
analyses after excluding participants who 
did not answer all the items in the question¬ 
naire, answer the 10 items about health be¬ 
haviors and perceptions, or those from the 
14-item CVD Knowledge Test. 26 The overall 
heart disease scale score was calculated by 
two different methods. We did these two 
analyses to show that the five domains are 
equally important to the total knowledge 
score. In Method I, the number of correct 
answers was counted and divided by the 
final number of items that resulted from 
the item reduction process (see below). In 
Method II, we calculated the overall score 
by taking the sum of the correct answers in 
each domain, dividing this number (correct 
items) by the total number of items in the 
domain. The result of this process gave us 
five percents (one for each domain), which 
were added together to give a total scale 


score ranging from 0 to 5. We expect that 
the results should be the same from these 
two different calculation methods. 

Analysis plan. The goal of Phase II was 
to validate the reduced item questionnaire 
resulting from Phase I and to validate the 
finalized questionnaire with other heart 
disease measures. 

SPSS Version 17 was used to obtain de¬ 
scriptive statistics on the reduced heart dis¬ 
ease knowledge questionnaire, the 10 items 
about health behaviors and perceptions, 
and the 14-item CVD Knowledge Test. 26 We 
obtained internal reliability estimates (coef¬ 
ficient alpha) of the finalized scale (obtained 
through item reduction), the five knowledge 
domains as well as correlations among the 
scale scores. The Kuder-Richardson formula 
20 for the internal reliability estimates was 
reported because our heart disease knowl¬ 
edge scale is dichotomous (true/false). 46 The 
correlations between the overall scale score 
on the heart disease knowledge question¬ 
naire and participants’ responses to the 
health literacy item were also calculated. 

The statistical software analysis program 
MPlus Version 5.21 was used to conduct 
the confirmatory factor analysis on the 
same five-domain reduced questionnaire. 
To assess model fit we reported the same fit 
statistics as in Phase I. 

RESULTS 

Phase I Questionnaire 

Descriptive statistics. For the 71-item 
scale, the average score was 35.80 with a stan¬ 
dard deviation of 8.87 and a range of 0 to 58. 
Fifty-four percent of participants answered 
36 or more of the items correctly. The item 
mean difficulty ranged from .13 to .93. 

Factor analyses. The one-factor confirma¬ 
tory factor analysis (CFA) solution had fit 
statistics of CFI = .76, TLI = .79, RMSEA 
= .04. The five-factor CFA solution had the 
same results of CFI = .76, TLI = .79, and 
RMSEA = .04. Given that neither model fit, 
we explored the modification indices. We 
found no modification in item loadings that 
would significantly improve model fit (all 
changes in yj were less than 4). 

An examination of the eigenvalues from 


an exploratory factor analysis (EFA) of the 
71 items revealed at least five factors. The 
first factor accounted for 15% of the total 
variance. Factors 1 through 5 accounted 
for 32% of the total variance. Because the 
sixth and seventh factors only accounted 
for a small amount of variance, 3% each, 
and did not provide additional interpre¬ 
tive solutions, we retained the five-factor 
model. The five-factor EFA model had good 
fit statistics with CFI = .96, TLI = .97, and 
RMSEA = .02. 

Based on the factor loadings of the 
71-item five-domain model CFA, 38 items 
loaded well onto one of the domains (fac¬ 
tor loadings > .40). From these 38 items, we 
discarded 17 items due to their difficulty 
mean (too easy or too difficult) and content 
(items’ topics were addressed in other items). 
For the 33 items that did not load well, we 
reviewed their content and mean difficulty 
level, and ultimately opted to retain 8 of 
these items, as their mean difficulties fell 
within the desired range and their content 
was deemed to be worthy of inclusion in the 
questionnaire. The item reduction of Phase 
I’s questionnaire produced 29 items. In sum, 
criteria used to get from 71 to 29 items were 
factor loadings, item content area, and dif¬ 
ficulty mean. 

Descriptive statistics: 29-item scale. The 
average score for the 29-item scale was 15.77 
with a standard deviation of 4.81 and a range 
of 0 to 28. Sixty-one percent of participants 
answered 15 or more of the items correctly. 
The item difficulty mean ranged from .18 
to .79. Correlations between the domains 
for the 29 items were low to moderate 
(Table 2). 

The 29-item questionnaire showed statis¬ 
tically significant associations with 3 items 
about self-perceived knowledge regarding 
heart disease: “I know more than the aver¬ 
age person about heart disease,” r(592) = 
.24, P < 0.01; “I believe that I answered 75% 
or more of the above questions correctly,” 
(r(592) = .25, P < 0.01); and “I believe that 
I answered more of the above questions cor¬ 
rectly than the average person will,”(r(592) 
= .37, P < 0.01). In other words, people 
who performed better on the scale tended 
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Table 2. Correlations Among the 5 Knowledge Domains (29 items) 

Variables 

Dietary 

Epidemiology 

Medical 

Risk Factor 

Symptoms 

Dietary 

- 

.25 

.37 

.38 

.25 

Epidemiology 


- 

.31 

.47 

.27 

Medical 



- 

.42 

.31 

Risk Factor 




- 

.33 

Symptoms 





- 

Note: All correlations significant at P < .001. 


to rate their performance more highly, 
although the moderate magnitude of these 
correlations suggests a fair degree of error 
in self-perceived knowledge. 

Phase II Questionnaire 

Descriptive analyses. By Method I calcu¬ 
lations (see Methods-Phase II section), the 
mean scale score for the 30-item scale (29 
items from Phase I, plus 1 rewritten item 
with the addition of “many of,” “Men and 
women experience many of the same symp¬ 
toms of a heart attack,” to improve clarity 
and included in the Phase II questionnaire) 
was 15.68 with a standard deviation of 4.71 
and a range of 2 to 25. Sixty-three percent 
of participants answered 15 or more of the 
items correctly. By Method II calculations, 
the mean scale score for the 30-item scale 
was 2.61 with a standard deviation of .80 and 
a range of .39 to 4.33. Because the 30-item 
scale was divisible by the scale score range 
of 5, these percentage scores were multiplied 
by 6, so that Method I and Method II final 
scale scores could be compared. Method 
II’s converted mean scale score was 15.67 
with a standard deviation of 4.79 and a 
range of 2.36 to 25.95. The item mean dif¬ 
ficulty ranged from .18 to .75. The Kuder- 
Richardson 20 formula for the 30-item scale 
was acceptable (.73), but was not for the 
five individual knowledge domains (range 
= .29 to .47). 

The mean scale score for the 10 items 
about health behaviors and perceptions was 
3.88 with a standard deviation of 1.57 and a 
range of 0 to 9. Approximately 59% of par¬ 
ticipants answered 4 or more items correctly. 


The mean scale score for the 14-item CVD 
Knowledge Test 26 was 7.70 with a standard 
deviation of 1.59 and a range of 2 to 12. 
Eighty percent of participants answered 7 
or more items correctly. 

The 30-item questionnaire showed sta¬ 
tistically significant associations with the 3 
items about self-perceived knowledge about 
heart disease (out of the 10 items about 
health behaviors and perceptions): “I know 
more than the average person about heart 
disease,” r(240) = .33, P < 0.01; “I believe 
that I answered 75% or more of the above 
questions correctly,” r(240) = .18, P < 0.01; 
and “I believe that I answered more of the 
above questions correctly than the average 
person will,” r(240) = .39, P < 0.01). Scores 
were also positively correlated with the item, 
“I know my blood pressure level,” r(240) = 
.15, P < 0.05. The correlation between the 
30 items and the 14-item CVD Knowledge 
Test 26 was moderately positive, r(240) = .59, 
P < 0.01, suggesting both convergent and 
divergent validity. The correlation between 
the 30 items and the health literacy item 
was weakly positive, r(240) = .32, P < 0.01, 
suggesting divergent validity. 

Factor analyses. The five-factor confirma¬ 
tory factor analysis (CFA) solution produced 
model fit statistics of CFI = .82, TLI = .88, 
and RMSEA = .03. Table 3 summarizes both 
the mean (difficulty) and factor loading 
(discrimination ability) of each item in each 
knowledge domain for the five-factor CFA. 
Appendix 1 presents the final 30-item heart 
disease knowledge questionnaire and in¬ 
cludes the item content, the correct answer, 


and the mean difficulty. 

Ad hoc panel. An ad hoc panel reviewed 
our final 30-item scale and revisions were 
done based on their feedback (Appendix 1). 
The panel suggested rewording three items, 
“Taking an aspirin each day is thought to 
decrease the risk of getting heart disease” to 
“Taking an aspirin each day decreases the risk 
of getting heart disease,” “The best kind of ex¬ 
ercise involves rapid breathing for a sustained 
period of time” to “The healthiest kind of ex¬ 
ercise for the heart involves rapid breathing for 
a sustained period of time,” and “Turning pale 
or gray is a common symptom of having heart 
attack” to “Turning pale or gray is a symptom 
of having heart attack.” Rewording the items 
did not change the correct response answer, 
nor do we think it influenced the results 
before the change. 

DISCUSSION 

This paper describes the development of 
a novel heart disease knowledge question¬ 
naire through two phases. The purpose of 
Phase I was to validate the five-factor model 
and perform item reduction. The result was a 
29-item questionnaire. The purpose of Phase 
II was to validate the final 30-item question¬ 
naire (29 items from Phase I along with an 
item that was rewritten and reintroduced). 
We also evaluated the extent to which par¬ 
ticipants accurately discerned the scope of 
their heart disease knowledge in both phases, 
and validated the new scale by correlating it 
with an existing but dated measure of heart 
disease knowledge, 26 as well as with a con¬ 
ventional measure of health literacy. 33 
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Table 3. Standardized Model Results— 

-Factor Loadings of the 5 Knowledge Domains (30 items) 

Items 

Dietary 
(N= 6] 

Epidemiology 

IN =4) 

Medical 
(N= 7) 

Risk 

Factors 

(N=9) 

Symptoms Difficulty 

(N - 4) Mean 

Polyunsaturated fats are healthier for 
the heart than saturated fats. 

28 




63 

Trans-fats are healthier for the heart 
than most other kinds of fats. 

.49 




.66 

Most of the cholesterol in an egg is in 
the white part of the egg. 

.56 




.65 

Dietary fiber lowers blood cholesterol. 

.44 




.48 

Margarine with liquid safflower oil is 
healthier than margarine with hydro¬ 
genated soy oil. 

.29 




.38 

Many vegetables are high in choles¬ 
terol. 

.29 




.71 

Women are less likely to get heart dis¬ 
ease after menopause than before. 


.54 



.34 

Heart disease is the leading cause of 
death in the United States. 


.44 



.60 

Most women are more likely to die 
from breast cancer than heart disease. 


.61 



.38 

Heart disease is better defined as a 
short-term illness than a chronic, long¬ 
term illness. 


.56 



.70 

Most people can tell whether or not 
they have hiqh blood pressure. 



.51 


.72 

The best kind of exercise involves rapid 
breathing for a sustained period of 
time. 



.29 


.38 

A healthy person's pulse should return 
to normal within 15 minutes after 



.41 


.73 

exercise. 






Cardiopulmonary resuscitation (CPR) 
helps to clear cloqqed blood vessels. 



.42 


.65 

HDL refers to "good" cholesterol, and 



.12 


D 1 

LDL refers to "bad" cholesterol. 





Atrial defibrillation is a procedure 
where hardened arteries are opened 
to increase blood flow. 



.48 


.19 

"High" blood pressure is defined as 

11 0/80 (systolic/diastolic) or hiqher. 



.37 


.26 

Having had chicken pox increases the 
risk of qettinq heart disease. 




.36 

.52 

Eating a lot of red meat increases heart 
disease risk. 




.20 

.65 

The most important cause of heart at¬ 
tacks is stress. 




.45 

.34 

Walking and gardening are considered 
types of exercise that can lower heart 
disease risk. 




.54 

.75 

Continues on next page 
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Table 3. Standardized Model Results—Factor Loadings of the 5 Knowledge Domains (30 items) (Con't) 

Items 

Dietary 
[N= 6) 

Epidemiology 
(IV =4) 

Medical 
(N= 7) 

Risk 

Factors 

(N = 9) 

Symptoms 
[N - 4) 

Difficulty 

Mean 

Smokers are more likely to die of lung 
cancer than heart disease. 




.44 


.23 

Taking an aspirin each day is thought 
to decrease the risk of getting heart 
disease. 




.44 


.70 

Taller people are more at risk for get¬ 
ting heart disease. 




.36 


.38 

People who have diabetes are at 
higher risk of getting heart disease. 




.43 


.64 

Eating a high fiber diet increases the 
risk of qettinq heart disease. 




.41 


.60 

Turning pale or gray is a common 
symptom of havinq heart attack. 





.44 

.51 

Sudden trouble seeing in one eye is a 
common symptom of having a heart 
attack. 





.47 

.18 

Feeling weak, lightheaded, or faint is 
a common symptom of having a heart 
attack. 





.58 

.71 

Men and women experience many of 
the same symptoms of a heart attack. 





.26 

.70 

Note: Standardized model results; STDYX Standardization 


Phase I Questionnaire 

The majority of the participants an¬ 
swered at least half of the items correctly for 
both the 71 and 29-item scales. Only about 
a quarter of the participants answered 2 or 
more of the 4 items about self-perceived 
knowledge in such a way as to reflect con¬ 
fidence in their knowledge of heart disease, 
both absolutely and in comparison to the 
average person. The item difficulty means of 
the 29 items fell within an appropriate range, 
with 11 items having a difficulty mean of .40 
to .60. The average difficulty mean for the 29 
items was .54. Scales with difficulty means 
that are close to .50 give a more accurate 
assessment of an individual’s knowledge 
level because they provide “maximum dis¬ 
crimination” 46 in individuals’ degree of heart 
disease knowledge. 

The CFA and EFA conducted on the 71 
items produced a reduced 29-item scale. 
Although the CFA produced mixed fit sta¬ 
tistics for our model, the EFA validated the 


decision that the five-factor model was a 
good choice for our 71-item scale. The cor¬ 
relations among the 5 knowledge domains 
for the 29 items were low to moderate, all 
positive and significant. These correlations 
indicate that the content information in 
each domain did not overlap, but that all 
domains were indeed measuring heart dis¬ 
ease knowledge. 

The correlation between the 29-item 
heart disease knowledge questionnaire and 
the 3 items assessing self-perceived knowl¬ 
edge was statistically significant, though 
moderate. Thus, people who know more are 
also generally more aware that they know 
more, but there is still a fair degree of error 
in these self-assessments. 

Phase II Questionnaire 

Both methods for calculating the overall 
score for the 30-item scale produced similar 
results. As with Phase I, the majority of par¬ 
ticipants answered at least half of the items 
correctly for the 30-item questionnaire. A 


majority of participants answered at least 4 
items on the 10-item scale about perceptions 
and health behaviors and at least 7 items cor¬ 
rectly on the 14-item CVD Knowledge Test. 26 
The level of difficulty was appropriate. 

The low alpha levels of the five knowl¬ 
edge domains suggest that while looking at 
how participants perform on each subscale 
of heart disease knowledge is informative, 
it cannot be considered a reliable scale for 
assessing group differences or changes over 
time. However, the overall 30-item scale 
had an acceptable internal reliability 47 of .73 
for examining group level scores on heart 
disease knowledge. 

The correlation between the 30-item 
scale and 14-item CVD Knowledge Test 26 
was moderately positive (.59) which sup¬ 
ports our assumptions that our current scale 
and the previous scale showed both conver¬ 
gent and divergent validity. It is evidence 
of convergent validity because it is 
associated with another scale tapping into 
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Appendix 1. Heart Disease Knowledge Questionnaire 

Instructions: 

On the following page, you will be asked to respond to a number of True/False questions addressing your beliefs and knowledge 
about various aspects of heart disease. 

Please answer each by circling "T" for True and "F" for False. 

Very few people answer all these questions correctly—just do the best you can. Feel free to circle 'Don't know' if you are unsure of 
an answer. 

EXAMPLE: 

High blood pressure increases the risk of getting heart disease.(?) F Don't know 

Item 

Item Content 

Correct 

Response 

Difficulty 

Mean 

HDI 

Polyunsaturated fats are healthier for the heart than saturated fats. 

True 

.63 

HD2 

Women are less likely to get heart disease after menopause than before. 

False 

.34 

HD3 

Having had chicken pox increases the risk of getting heart disease. 

False 

.52 

HD4 

Eating a lot of red meat increases heart disease risk. 

True 

.65 

HD5 

Most people can tell whether or not they have high blood pressure. 

False 

.72 

HD6 

Trans-fats are healthier for the heart than most other kinds of fats. 

False 

.66 

HD7 

The most important cause of heart attacks is stress. 

False 

.34 

HD8 

Walking and gardening are considered types of exercise that can lower heart disease risk. 

True 

.75 

HD9 

Most of the cholesterol in an egg is in the white part of the egg. 

False 

.65 

HDI 0 

Smokers are more likely to die of lung cancer than heart disease. 

False 

.23 

HDI 1 

Taking an aspirin each day decreases the risk of getting heart disease. 

True 

.70 

HDI 2 

Dietary fiber lowers blood cholesterol. 

True 

.48 

HD13 

Heart disease is the leading cause of death in the United States. 

True 

.60 

HD14 

The healthiest exercise for the heart involves rapid breathing for a sustained period of time. 

True 

.38 

HDI 5 

Turning pale or gray is a symptom of having a heart attack. 

True 

.51 

HD16 

A healthy persons pulse should return to normal within 15 minutes after exercise. 

True 

.73 

HDI 7 

Sudden trouble seeing in one eye is a common symptom of having a heart attack. 

False 

.18 

HD18 

Cardiopulmonary resuscitation (CPR) helps to clear clogged blood vessels. 

False 

.65 

HD19 

HDL refers to "good" cholesterol, and LDL refers to "bad" cholesterol. 

True 

.31 

HD20 

Atrial defibrillation is a procedure where hardened arteries are opened to increase 
blood flow. 

False 

.19 

HD21 

Feeling weak, lightheaded, or faint is a common symptom of having a heart attack. 

True 

.71 

HD22 

Taller people are more at risk for getting heart disease. 

False 

.38 

HD23 

"High" blood pressure is defined as 110/80 (systolic/diastolic) or higher. 

False 

.26 

HD24 

Most women are more likely to die from breast cancer than heart disease. 

False 

.38 

HD25 

Margarine with liquid safflower oil is healthier than margarine with hydrogenated soy oil. 

True 

.38 

HD26 

People who have diabetes are at higher risk of getting heart disease. 

True 

.64 

HD27 

Men and women experience many of the same symptoms of a heart attack 

True 

.70 

HD28 

Eating a high fiber diet increases the risk of getting heart disease. 

False 

.60 

HD29 

Heart disease is better defined as a short-term illness than a chronic, long-term illness. 

False 

.70 

HD30 

Many vegetables are high in cholesterol. 

False 

.71 

Note: Items HD4 and HD28 overlap with both dietary and risk factor domains. However, we kept them in the risk factor domain because they refer to a par¬ 
ticular dietary risk factor of heart disease, as opposed to general dietary aspects pertaining to overall heart health, which are in the dietary domain. 
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similar heart disease knowledge. However, 
it also can be said that there is unique in¬ 
formation contained in our current scale 
as it only has about 36% shared variance 
with the other scale. Finally, the correlation 
between the 30-item scale and the 1 item 
about health literacy was weakly positive. 
The correlation between the 30-item scale 
and the item about knowledge of blood 
pressure level was found to be significant, 
though low, suggesting that people who 
are aware of their medical status are not 
necessarily more knowledgeable about 
heart disease in general. This suggests 
divergent validity because the 1 item mea¬ 
sure of health literacy cannot adequately 
measure knowledge in a given area of health 
knowledge and that participants were often 
inaccurate in their judgments of their heart 
disease knowledge. 

Although the five-factor CFA model fell 
below the good fit criteria of CFI and TLI, 
the loadings were relatively good. Twenty- 
one of the items had factor loadings above 
.40, which shows that the items loaded well 
onto their pre-established factors. 

Limitations 

One limitation of the current study is 
the use of a university student sample. De¬ 
pending on individual circumstances, heart 
disease may not be a relevant issue for this 
particular population; thus, they may be less 
educated about it than an older population. 
To determine if there was a difference in 
knowledge level based on age and years of 
education, we compared the 30-item overall 
scale scores between the current Phase II 
sample (N = 242) to a sample of adults. Par¬ 
ticipants were 27 adults from the Pittsburgh 
community who had completed the heart 
disease knowledge questionnaire as part of 
another study. The participants were 66.7% 
female with a mean age of 56.93 (SD = 9.62, 
range = 40 to 79). For the adult sample, the 
mean 30-item scale score was 12.56 with a 
standard deviation of 3.84 and a range of 4 
to 18 (as compared with the college sample, 
M = 15.68, SD = 4.71, range = 2 to 25). 

The correlations between the overall 
score and age, and the overall scale score 
and years of education were calculated. The 


correlation between the overall score and 
age was weakly negative, but statistically 
significant, r(267) = -.17, P < 0.01. There 
was no correlation between overall score 
and years of education, r(264) = -.03, P > 
0.05. Overall, these findings argue against 
the concern that the young age of the sample 
was problematic. 

In addition, neither the Phase I nor 
Phase II samples were asked to specify race/ 
ethnicity. This makes it impossible to ex¬ 
amine the results for differences in levels of 
knowledge among those of varying racial/ 
ethnic groups. Future studies are needed to 
determine whether the results of the current 
study can be generalized to other popula¬ 
tion samples, and to explore the impact of 
individuals’ race or ethnicity on level of heart 
disease knowledge. 

The different methods used in Phase I 
and Phase II to administer the heart disease 
questionnaire is a possible limitation. In 
Phase I the questionnaire was completed 
as part of a mass testing session, but in 
Phase II the questionnaire was completed 
privately on a computer. It is possible that 
there maybe potential method effects from 
this difference. However, a reliable and valid 
questionnaire should be able to be used in 
multiple contexts. 

Another limitation is the manner in 
which we chose to reduce items in Phase I 
analyses. It is possible that through discard¬ 
ing items based on their content validity we 
removed items that might in fact be impor¬ 
tant to the overall construct of heart disease 
knowledge, though in the present study met 
all criteria for removal. 

A final limitation is that the variance 
from the Phase 1,71-item EFA only account¬ 
ed for about 30% of the total variance. One 
reason for this low variance is that we were 
only interested in the participants’ overall 
scores as opposed to the profile scores of 
the individual knowledge domains. Both the 
overall score and the profile score have their 
benefits. The overall score may account for 
a lower variance, but this score can help to 
develop heart disease prevention education 
policy because the overall score provides a 
single score that can be used to compare 


different samples. On the other hand, the 
profile score may produce better vari¬ 
ance, but can only help to develop tailored 
interventions for that specific individual 
based on how they did on each domain. We 
recommend that further researchers look at 
both the overall score and profile score to 
determine a more accurate assessment of 
a scale’s variance. Doing this may also help 
to shape broad policy changes, as well as 
tailored individual interventions in the field 
of heart disease prevention education. 

Heart disease is the number one killer of 
both men and women in the United States. 
Knowledge of risk factors, symptoms, and 
epidemiology of heart disease is essential to 
prevention, identification, and appropriate 
action, yet a comprehensive and evidence- 
based knowledge assessment is currently not 
available. This current study met this need 
by developing a heart disease knowledge 
questionnaire. Overall, the results indicate 
that the resulting 30-item questionnaire de¬ 
signed to measure heart disease knowledge, 
as conceptualized across five key domains, 
exhibited sound psychometric properties. 

Designed to be easily administered, this 
scale could serve as a valuable tool to re¬ 
searchers and practitioners alike. By includ¬ 
ing items with a wide range of difficulties, it 
possesses maximum power to discriminate 
across the spectrum of knowledge levels, and 
could likewise be useful in the identification 
of knowledge gaps at an individual level, so 
as to inform tailoring of health informa¬ 
tion. It is also more comprehensive than any 
existing scale in that it measures knowledge 
of heart disease etiology, epidemiology and 
symptomatology and can be used with most 
adult samples. 

One interesting finding is that partici¬ 
pants’ perceived knowledge was somewhat 
incongruent with their actual knowledge. 
Studies in such areas as condom-use be¬ 
havior 48 ' 49 and food-related hazards 50 have 
shown that people who rate their perceived 
knowledge as high often actually have 
low levels of actual knowledge. Thus, it is 
particularly essential to use actual scales to 
measure heart disease knowledge rather than 
to rely on self-reports. 
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One of the primary challenges of ques¬ 
tionnaire development is to ensure that the 
essential aspects of a particular construct 
are captured, and we believe that the items 
in the current questionnaire do indeed ad¬ 
dress the central elements of heart disease 
knowledge. We considered many criteria in 
order to determine item reduction includ¬ 
ing factor loadings, reliability, items’ content 
areas, and a review by an ad hoc panel of 
the resulting scale items. This allowed us 
to ensure that the scale’s ultimate content 
was sufficiently reflective of each important 
knowledge domain. 

TRANSLATION TO HEALTH 
EDUCATION PRACTICE 

Findings from this study are consistent 
with the growing heart disease literature, 
describing a gap between perceived and 
actual knowledge of heart disease. 28,30,51 ' 53 
For example, the low correlation between 
the one item addressing medical status and 
the 30-item overall scale score suggests that 
awareness of personal health biomarkers, 
such as blood pressure, is not reliably as¬ 
sociated with breadth of knowledge about 
the causes, symptoms and other attributes of 
heart disease. In addition, the weak correla¬ 
tion between the measure of health literacy 
and the 30-item overall scale score indicates 
that variation in knowledge about heart 
disease cannot be accounted for simply by 
differences in health literacy. 

The current scale could be used by health 
professionals such as cardiologists, primary 
care providers, and health educators to tailor 
educational programs to the knowledge base 
of their individual patients. These programs 
would help meet the needs of patients by 
helping to bridge gaps between perceived 
and actual knowledge of heart disease. Ben¬ 
efits of tailored education materials include 
an increased likelihood that individuals will 
have read the information, 55 ' 56 remembered 
it, 55 ’ 56 and will find it personally relevant. 55 
Moreover, tailored health education/infor¬ 
mation has been shown to reduce health 
literacy disparities 57 and works well in the 
context of other intervention programs. 55 
Tailored programs could ensure that indi¬ 


viduals are not only becoming more knowl¬ 
edgeable about heart disease, but that they 
know which specific behaviors and actions 
they can take to reduce their risk, such as 
changing diet, increasing amount of physi¬ 
cal activity, and being aware of signs and 
symptoms of a heart attack. 

The goal of having individuals complete 
this scale and review their score with a health 
educator is that if individuals know their 
heart disease knowledge score, are aware 
of their knowledge gaps, and understand 
the differences between their perceived and 
actual knowledge, then they will be more 
likely to apply their increased heart disease 
knowledge to behavior change. Although 
increased knowledge does not always lead 
to behavior change, 52 ’ 54 a review of tailored 
print communications studies showed 
that individualized tailored information 
can lead to positive behavioral change. 55 
Tailored health education programs that 
address heart disease knowledge gaps have 
the potential to lead to change in areas such 
as nutrition, exercise, and smoking, 55 all 
risk factors for heart disease. The tailored 
education programs, in conjunction with 
other intervention programs, may lead to 
more effective and efficient patient care for 
those at risk of heart disease. 
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